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ABSTRACT 

Inherently a discovery-based pedagogy, Disassemble/Analyze/Assemble (DAA) activities start with 
the artefact, an instance of a typically well-engineered solution. Through systemized disassembly and 
the subsequent analysis of components, students engage in an iterative process of observation and 
follow-up probing. In-turn, this process helps students understand the function of the artefact’s com¬ 
ponents and their interconnection with each other and the operation of the artefact. Previous studies 
have provided highly descriptive accounts of curricula outcomes of DAA activities; but relatively few 
have compared participants doing DAA activities to a control group learning the same content in a 
more traditional fashion. To address this issue, a quasi-experiment was conducted as part of a first-year 
engineering laboratory, where a DAA activity was compared to a lecture on the same content. The 
results showed that students who engaged in the DAA activity were more motivated and demonstrated 
higher frequencies of transfer than those receiving lecture. Superior transfer by the DAA condition 
was found even after controlling for prior knowledge of the transferrable element. 

Key Words: disassembly, transfer, motivation, discovery-based pedagogy 


INTRODUCTION 

To achieve mastery of the fundamental ideas of a field, Bruner suggests that in addition to grasp¬ 
ing the general principles, one must also “develop an attitude towards learning and inquiry, towards 


SUMMER 2013 


1 




ADVANCES IN ENGINEERING EDUCATION 

The Relative Pedagogical Value of 
Disassemble/Analyze/Assemble (DAA) Activities 


guessing and hunches, towards the possibility of solving problems on one’s own” (1977, p. 20) 
Although uncertain at the time about the required approach for instilling these crucial attitudes in 
learners, Bruner expressed a lack of confidence in the adequacy of the mere presentation of ideas. 
Instead, he pushed for the inclusion of discovery, the process whereby regularities of previously 
unrecognized relations and similarities between ideas are uncovered by the learner, resulting in a 
sense of self-confidence in one’s abilities. Advocates of discovery learning tend to support Piaget’s 
assertion that “each time one prematurely teaches a child something he/she could have discov¬ 
ered for himself/herself, that child is kept from inventing it and consequently from understanding 
it completely” (Piaget, 1970, p. 715). Following the publication of Bruner’s Art of Discovery (1979), 
a flurry of research on discovery based pedagogical methods ensued (Mayer, 2004). Many stud¬ 
ies entailed comparisons among pure discovery methods where students were required to solve 
problems with little or no guidance; guided discovery methods where hints, direction, coaching, 
feedback, and/or modeling were provided to keep the student on track; and expository methods 
where students were given the problem along with the correct answer. Scholars such as Schwartz, 
Bransford, and Sears (2005) found that innovation or discovery-oriented activities are instrumental 
to deep understanding and transfer. Others, such as Klahr and Nigam (2004) and Kirschner, Sweller 
and Clark (2006), argue in favor of direct instruction, finding that discovery learning simply takes 
longer, mainly benefits top-achievers or those with sufficiently high prior knowledge, and can lead 
to the development of misconceptions. 

In spite of the ongoing debate, active engagement and motivation have emerged as undisputed 
features of discovery-based learning; however, one of the main sources of contention lies in the 
inability of the approach to constrain students’ exploration without guidance. What students self- 
discover may not always be what was intended to be taught. For example, actively constructing 
objects is not necessarily the same as constructing an understanding of how they work. Students may 
demonstrate their ability to make a model rocket but develop little understanding of aerodynamics. 
The concern then is that discovery activities take time and may not provide sufficient parameters 
to direct students’ attention to topic-relevant concepts. 


DISASSEMBLE/ANALYZE/ASSEMBLE (DAA) ACTIVITIES - REVERSE DISCOVERY 

Inherently a discovery-based pedagogy, Disassemble/Analyze/Assemble (DAA) activities start 
with the artefact, an instance of a typically well-engineered solution. Through systemized disassembly 
and the subsequent analysis of components, students engage in a potentially self-directed iterative 
process of observation and follow-up probing. In turn, this process helps students understand the 
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function of the artefact’s components and their interconnection with each other and the operation 
of the artefact. Typical discovery activities such as inventing or construction tend to be flawed by 
their inability to constrain students’ explorations and prevent deviation from the intended focus. DAA 
activities attempt to overcome this challenge by starting with the expert version, an approach that 
has shown success at facilitating learning, transfer, and motivation (Dalrymple, Sears, & Evangelou, 
2011 ). 

DAA activities are well suited for engineering education given the relationship between engineer¬ 
ing and artefacts. Engineering thinking is materialized largely through interactions with artefacts, the 
material aspects of our physical world. Engineers interact with artefacts as creators, bringing them 
into existence; curators, maintaining and furthering them; and controllers, extending knowledge 
and access of and to these artefacts. Studying the pedagogical affordances of this type of artefact 
interaction (i.e., DAA Activities) can continue to extend our understanding of discovery-based 
learning pedagogies with the additional potential to reveal new ways in which we can specifically 
nurture engineering thinking. 


DAA ACTIVITIES IN ENGINEERING EDUCATION 

The application of DAA activities in engineering learning environments has generated rave re¬ 
views from both instructors and students. Starting in 1991 with the work by Sheri Sheppard in her 
mechanical engineering dissection course at Stanford University, the following learning outcomes 
have been associated with the use of DAA activities: helping students identify relationships be¬ 
tween theoretical concepts and their real-world instantiations (Brereton, Sheppard, & Leifer, 1995), 
increasing motivation and retention (Carlson, Schoch, Kalsher, & Racicot, 1997), encouraging the 
development of curiosity, proficiency and dexterity (Beaudoin & Ollis, 1995; Hess, 2002), providing 
hands-on activities to couple engineering principles with significant visual feedback (Barr, Schmidt, 
Krueger, & Twu, 2000; McKenna, Chen, & Simpson, 2008), and supporting design learning (Deven- 
dorf, Lewis, Simpson, Stone, & Regli, 2007; Ogot, Okudan, Simpson, & Lamancusa, 2008; Wood, 
Jensen, Bezdek, & Otto, 2001). 

One of the noted shortcomings of the current literature on DAA activities is the lack of empirical 
evidence from controlled experiments that indicate the advantages of DAA pedagogies over other 
traditional forms of instruction. In response to this need, recent explorations conducted by the au¬ 
thors into the pedagogical viability of DAA activities, have experimentally confirmed the potential 
of DAA activities to elicit motivation over more traditional forms of instructions (i.e., step-by-step 
lab instructions) and identified its additional benefit to promote transfer to novel design problems 
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(Dalrymple et aI., 2011). A follow-up study, which is described in this publication, extends on the 
authors’ initial findings by further isolating the factors in DAA activities that are instrumental to 
students’ learning and motivation in engineering. 


CONTRIBUTIONS OF THE STUDY 


In a previous experiment, a DAA activity was compared to a different engineering technique for 
their respective abilities to foster motivation and transfer of learning (Dalrymple et al., 2011). Results 
revealed that the DAA activity promoted greater motivation and transfer, but, relevant to the cur¬ 
rent study, it also pushed for learning different content. The DAA activity supported the learning 
of specific part-function relationships while the control condition supported greater breadth. This 
raised the concern that if the content that was emphasized by each condition was different, the 
results for the DAA activity could simply be due to its attention to part-function relationships rather 
than what we consider to be the hallmark of DAA activities—direct manipulation of an artefact. In 
addition, the previous study only used one measure of transfer and a three-question measure of 
motivation, so the degree of generalizability of the findings required further testing. In the current 
study, a DAA activity is compared to a direct method of instruction i.e., lecture. Both instructional 
methods are designed to help students learn the same content knowledge because both address 
part-function relationships. This improved comparability from the previous study allows for evalua¬ 
tions on multiple dimensions of learning (e.g., factual recall of part-function relationships and mul¬ 
tiple measures requiring redesign or knowledge transfer). With this level of control, any resulting 
significant differences between conditions suggest benefits inherent to the manipulative process 
in DAA activities. In addition, well-established measures of motivation were added to the previous 
measures to ensure a more complete picture of the effects of DAA activities. 


DAA VERSUS DIRECT INSTRUCTION 


In the first U.S. edition of Donald Bligh’s “What’s the Use of Lectures?”(2000), a lecture is defined 
as “a period of more or less continuous exposition by a teacher.” Bligh also provides the outcomes 
of numerous experimental comparisons to substantiate the claim that lectures are as effective as 
other methods of instruction for transmitting information. He also goes on to validate in the same 
manner that lectures are not as effective as discussion methods for the promotion of thought, and 
relatively ineffective for inspiring interest in a subject. Thought as described by Bligh can be likened 
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to the cognition or deeper understanding required to enable transfer to novel problems, and inter¬ 
est in a subject matter likened to motivation. A key difference between lecture and DAA methods 
of instruction is that even if the factual content is the same, the DAA method is expected to afford 
more opportunities for students to think about the interconnection of parts and functions because 
they must attend to these while disassembling an artefact, especially if the goal is to understand 
the design of the artefact. In this sense, the manipulation of the artefact and the learning of part- 
function relationships are directly interconnected in DAA, perhaps lending greater meaningfulness 
and motivation to the DAA activity. Hence it is within reason to expect the outcomes achieved in the 
previous experiment (Dalrymple et a I., 2011) to persist. For the current study it was hypothesized 
that on measures of motivation, the DAA activity will be rated higher than the lecture; on measures 
of learning where the task required students to recall the part-function relationships explored in 
both instructional approaches, the DAA activity and lecture will result in equivalent performance; 
and on measures of deeper understanding, like the application of knowledge to redesign or defect 
diagnosis tasks, the DAA activity will result in greater transfer. 


METHODS 

The study was conducted within the context of a laboratory, following approval from the Institu¬ 
tional Review Board. The laboratory was designed to introduce first-year engineering students to 
the principles of design for the environment (DfE) through the study of a Fujifilm single-use camera. 
DfE refers to the systematic consideration of design performances with respect to environmental, 
health, and safety objectives over the full product life cycle. Fujifilm applies these principles in the 
design and development of its line of single-use cameras. The single-use cameras are produced in 
an inverse manufacturing facility where 99% of used cameras are either remanufactured or recycled 
to produce new generations of the product. Both a lecture and a DAA activity were utilized to help 
students learn about the design of the camera. With the DAA activity, students disassembled the 
camera and analyzed its components to discover their function and interconnectedness, while the 
lecture presented similar content with the use of a multimedia PowerPoint presentation. To measure 
and compare the learning outcomes from each instructional method, yet ensure all students had an 
equivalent learning experience, the lab was completed in one of two sequences: 162 students did the 
DAA activity before the lecture (Sequence 1 - DAA First) and 163 students had the lecture before the 
DAA activity (Sequence 2 - Lecture First). Assessment activities, administered online, and completed 
by each student individually, preceded and followed each instructional method. The assessments 
were used to measure the extent to which the DAA activity and the lecture facilitated learning and 
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Duration 

Task# 

Sequence 1 
(DAA First) 

Sequence 2 
(Lecture First) 

Task Type 

8 min 

- 

Introduction 

Class 

10 min 

1 

Pretest - Match Components to Functions 

Individual 

30 min 

2 

DAA Activity 

Lecture 

Team/Class 

20 min 

3 

Posttest 1 - Variant Design, System 
Decomposition & Camera Doctor 

Individual 

30 min 

4 

Lecture 

DAA Activity 

Class/Team 

5 min 

5 

Posttest 2 - Unwanted Features 

Individual 

7 min 

- 

Post Lab Survey 

Individual 


Table 1. Study design/task sequencing. 


transfer of knowledge about the function and interconnectedness of the components used in the 
camera’s design. To ensure consistency in the facilitation of the lab, particularly the presentation 
of the lecture, all labs were led by the same person. Table 1 shows the design of the study and the 
sequence of tasks in the lab. Each lab task is described in the following sections. 

Introduction 

The lab began with an 8-minute PowerPoint presentation. In the presentation, DfE was described 
and demonstrated in the design and development of Fujifilm’s line of QuickSnap single-use cameras. 
Students were also given an overview of the lab tasks to follow and procedural instructions for ac¬ 
cessing the online assessments. 

Task 1 (Pretest) 

The first task was the same in both sequences, and it served as the pretest. It provided a measure 
of student’s prior knowledge of the part-function relationships of the single-use camera that was 
studied in the lab. For the task, students were asked to match the components of the camera to 
their functions. Each of the eight questions contained a different function description and pictures 
of eight components from the Fujifilm QuickSnap Outdoor 1000 single-use camera. Students were 
instructed to select the component that fulfilled the described function. 

Task 2 

The second task was either the DAA activity (Sequence 1 / DAA First) or the lecture (Sequence 
2 / Lecture First). For the DAA activity, students worked in teams of 3-4 students. Each team was 
given an instruction sheet that introduced the concept of reverse engineering and its application 
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in industry as a tool used by companies to compare their services and practices to competitors. 
Also included was a description of a fictitious company interested in developing a line of single-use 
cameras that wanted to learn more about the design of the Fujifilm single-use camera. Students 
were asked to assist in this venture by working in teams to systematically disassemble a Fujifilm 
QuickSnap Outdoor 1000 camera. They were also asked to use any representation of their choice 
to record the camera components along with a description of how each component functions to 
make the camera work. With the exception of a few hints to help start the disassembly process, 
student teams approached this task in their own way, probing their observations to the extent they 
deemed necessary to understand the camera’s design. 

In the lecture, the design of the Fujifilm QuickSnap Outdoor 1000 camera was presented using 
a multimedia PowerPoint presentation. The presentation began with a brief history of one-time use 
cameras and the development process used by Fujifilm. Following this, all components of the cam¬ 
era, starting with the packaging, were identified, and their functions described with the use of one 
or a combination of the following: animation, computer-aided design (CAD) renditions, and video 
clips showing the movement of components within the camera and their interconnection with other 
components. The components were presented in the order one may encounter them if disassembling 
the camera. Figure 1 shows excerpts of the PowerPoint slides used for the lecture. 

Task 3 (Posttest 1) 

The third task was the same in both sequences, and it served as the first posttest. Using an 
instrument administered online, students were asked to respond to three types of questions 
(i.e., system decomposition, camera doctor and variant design,). The three question types relate 
to common tasks performed in the engineering design process. All questions were designed to 
apply the knowledge students were expected to have gained about the camera’s part-function 
relationships from the previous task (DAA activity or lecture). A description of the three question 
types follows: 

• System decomposition refers to the process of dividing a system into smaller parts or subsys¬ 
tems for the purpose of reducing complexity. It occurs in the problem definition phase of the 
design process to facilitate a better understanding of the problem to be solved. A system can 
be decomposed based on functions, user actions or key customer needs (Ulrich and Eppinger, 
2004). For the system decomposition component a different user or camera action (i.e., aim, 
shoot, wind, and protect film) was described in each of the four questions. Also included in 
each question were the pictures of 18 Fujifilm QuickSnap Outdoor 1000 camera components. 
Students were required to select all the components that function to allow each identified 
action to occur. 
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Overall Structure Comprises: 

• Front and Back Cover - (Recycled) 

• Main Body - (Reused via Remanufacturing) 
>Mechanical Parts 

• Film (Returned to User) 

99% Re-used & Recycled 



Figure 1. Excerpts from the multimedia PowerPoint presentation used with the lecture. 


• Troubleshooting is the common engineering task that students were asked to perform with 
the Camera Doctor questions. It is a form of problem solving that is applied to the repair of 
failed products or processes, and is very prevalent in the testing phase of the design process. 
Troubleshooting requires an integrated understanding of how the system being troubleshot 
works (Jonassen, 2000). For the camera doctor component, examples of photographs taken 
with cameras with different defects were presented in each of the four questions. Table 2 shows 
examples of the photographs that were used in the questions. Also, included in each question 
were the pictures of 18 Fujifilm QuickSnap Outdoor 1000 camera components. Students were 
required to diagnose the defect that would have led to the poor photograph by choosing the 
malfunctioned user or camera action (aim, shoot, wind, or protect film), selecting the com¬ 
ponents that may be defective from the 18 components presented, and describing what may 
have gone wrong in terms of the camera’s functionality. 

• Variant design is one classification of engineering design. It involves varying the parameters of 
certain aspects of a product to develop a new and more robust design (Otto & Wood, 2001). 
Variant design techniques are used to create scaled product variations for a product line. For 
the variant design component two scenarios were presented, each describing a need for new 
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Pictures Description 


The picture has a dark rounded spot in the top left hand comer. The image 
progressively fades and becomes more blurred towards the bottom right 
hand comer. 


The picture below has discolored streaks along the sides. 


Table 2. Examples of the photographs used in the camera doctor questions. 



camera functionality (i.e., three different shutter speeds and a viewfinder that can be used to 
see any object blocking the lens). Students were asked to describe how the original design 
of the camera could be modified to achieve the new functionality. 

Task 4 

The fourth task was either the lecture (Sequence 1) or the DAA activity (Sequence 2), a reversal 
of task 2 such that all participants would experience both instructional methods. 

Task 5 (Posttest 2) 

The fifth task was the same in both sequences, and it served as the second posttest. For this task 
students were presented with images of 18 Fujifilm QuickSnap Outdoor 1000 camera components 
and asked to select from this group all the components that function to prevent the superimpos¬ 
ing of images (i.e., multiple pictures being captured on the same film frame). This task, like task 3, 
was designed to apply the knowledge students were expected to have gained about the camera’s 
part-function relationships. Following both instructional tasks, this assessment allowed the effect 
of different task sequences to be evaluated. 

Post Lab Survey 

Following task 5, students completed a post lab survey. Students responded to questions about 
their background and perception of both instructional tasks. Using the seven-point Likert-type 
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scale used in the previous study (Dalrymple et al., 2011), students rated both the DAA activity and 
lecture on: 1) perceived sense of learning, 2) enjoyment derived from engaging in the activity, and 
3) helpfulness in preparing them to respond to the questions in task 3. The three aforementioned 
elements were used to measure the motivation elicited from each instructional approach. Students 
also rated their prior experience disassembling objects using a seven-point scale that ranged from 
(1) no experience to (7) extensive experience. To further measure the motivation specifically elic¬ 
ited by the DAA activity, 30 questions from the Intrinsic Motivation Inventory (IMI) were used to 
capture students’ interest, perceived competence, effort, value and pressure experienced while 
engaging in the DAA activity. The IMI is a multidimensional measurement device designed to as¬ 
sess participants’ subjective experience related to a target activity (Deci, Eghrari, Patrick, & Leone, 
1994; Ryan, 1982). 


ANALYSIS 


The unit of analysis was the individual. The pretest, posttest 1, posttest 2, and post lab survey 
were scored and analyzed for each student to: 

• Measure and compare the frequency of students that transferred knowledge from the DAA 
activity and lecture. 

• Measure and compare students’ reported sense of learning, enjoyment, and perception of 
helpfulness (elements of motivation) for the DAA activity and lecture. 

• Measure students’ interest, perceived competence, effort, value and pressure experienced (IMI 
measures) while engaging in the DAA activity. 

• Identify demographic, background, and motivation measures that may potentially mediate 
the transfer of knowledge from the DAA task. 

To established reliability of scoring on open-ended questions in posttest 1, a randomly selected 
subset, consisting of 15% of the responses to variant design and camera doctor questions were inde¬ 
pendently scored by two researches. A minimum of 90% consensus was attained on both question 
types. All other questions were scored using a developed Microsoft Excel formula that was applied 
to all responses in the sample. Details of the scoring and analysis procedures used for each assess¬ 
ment instrument are provided in the following sections. 

Pretest 

Responses to the pretest were evaluated to determine students’ prior knowledge of the part-func¬ 
tion relationships of the Fujifilm single-use camera. Students were asked to select the components 
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that matched the identified function descriptions. Every correct part-function match was awarded 
one point. The maximum score attainable for the pretest was eight. The average pretest score of 
students in each sequence, i.e., DAA First or Lecture First, was statistically compared to test for 
condition comparability at the start of the lab. It was expected that all student will show little prior 
knowledge of the camera’s part-function relationships. 

Posttest 1 

The three types of questions used in posttest 1 were expected to illuminate the differences in the 
two instructional approaches (DAA activity and lecture) in terms of learning. Both instructional ap¬ 
proaches were expected to have equivalent outcomes on the system decomposition questions which 
mainly required students to recall information about the functions of the camera’s parts. The DAA 
activity, however, was expected to show advantages over the lecture on the variant design and camera 
doctor questions, which required the knowledge of the function and interconnectedness of the parts 
to be recalled, adapted and applied to novel problems. To test these hypotheses the average scores 
of students in the DAA First and Lecture First conditions were compared for each question type. 

The system decomposition questions asked students to identify all the components, from a selec¬ 
tion of 18, that function to allow each described action to occur. Students were awarded a point for 
each correct component selected and each incorrect component that was not selected. No points 
were awarded for selecting or not selecting components that do not have a primary contribution 

to the identified action. To ensure a maximum attainable score of 1, a point was valued at-]-, 

(18 - A/PC) 

where NPC is the number of components that do not have a primary contribution to the identified 
action. An example of this scoring method is presented in Table 3. 

For the camera doctor questions, students were required to diagnose what part-function failures 
would cause each defective photograph at three levels of specificity by: 1) choosing the malfunc¬ 
tioned user or camera action; 2) selecting possible components, associated to the malfunctioned 
user or camera action, that may be defective; and 3) identifying the main defective component and 
describing how it may have malfunctioned. For each correctly identified user or camera action, a 
score of 1 was assigned, otherwise the score was 0. The selection of possible defective components 
was scored in the same way as the system decomposition questions. With respect to the descriptions 
of how the main component malfunctioned, answers that identified the correct primary mechanism 
with defects received a score of 1 and those that did not received a score of 0. The description com¬ 
ponent of three of the camera doctor questions was scored. The description component of the fourth 
question was omitted since the presented scenario could have been caused by multiple primary 
components with defects and a reliable scoring method could not be achieved. Examples of scored 
responses that described how the main component malfunctioned are presented in Table 4. 
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Components labeled F, O and R have a primary 
contribution to the Aim action. For identifying these 
components 3 points are received (one point each). 
Components A, B, C, D, E, H, I, J, L, M, N, P, and 
Q do not relate to the Aim action and 13 points are 
received for not identifying these components (one 
point each). Component G is interconnected with F, and 
component K is interconnected with components O and 
R, however G, and K do not have a primary contribution 
to the Aim action. No points are awarded for not 
selecting them. 

Total points received =16 

1 point =_ l _=_ l _= _L 

18 -NPC 18-2 16 

Total Score = 16 x — = 1 
16 


Table 3. Scoring method for system decompositions questions. 


Photo 


Response Scoring Rationale Score 



The picture has a dark rounded spot in the top left 
hand corner. The image progressively fades and 
becomes more blurred towards the bottom right 
hand corner. 


The shutter’s spring may 
have been worn down and 
not pulled the shutter 
back smoothly, or the 
shutters parts may have 
caught and not opened 
and closed smoothly. 

Film was exposed to 
too much light on one 
side and not enough on 
the other. It most likely 
wasn’t flat against the 
inside. 


The correct primary 
mechanism with defects 
is identified. 


The correct primary 
mechanism with defects 
is not identified. 


1 


0 


Table 4. Example of scored responses to the camera doctor questions. 
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For the variant design questions students were asked to describe how the original design of the 
camera can be modified to achieve: 1) three different shutter speeds and 2) a viewfinder that can 
be used to see any object blocking the lens. A reasonable approach for achieving three different 
shutter speeds includes a modification to the extension spring system. Descriptions that included 
a modification to the spring system received a score of 1, and those that did not received a score 
of 0. To make it possible for the viewfinder to be used to see any objects blocking the lens the rays 
of light entering the lens must be directed to the viewfinder. Figure 2 shows the implementation of 
the viewfinder and lens in the design of the Fujifilm single-use camera. Descriptions that described 
an appropriate mechanism such as a mirror or prism for accomplishing this received a score of 1. 
Descriptions that indicated the need for a mechanism to direct the rays of light entering the lens 
to the viewfinder, without identifying the mechanism received a score of %. Descriptions that sug¬ 
gested the lens and viewfinder should be superimposed received a score of K (although this answer 
alludes to the fact that rays of light entering the lens should also enter the viewfinder, this solution 
is infeasible). Descriptions that did not mention or suggest that the lens and viewfinder need to be 
interconnected received a score of 0. Examples of scored responses to the variant design questions 
are presented in Table 5. 


EXTERNAL VIEW INTERNAL VIEW 



Figure 2. Viewfinder and lens in Fujifilm QuickSnap Outdoor 1000. 
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Response 

Scoring Rationale 

Score 

Add a selection knob to select the tension on the shutter spring. Set 
three different shutter spring tensions so you will have three different 
speeds. 

Spring system is modified 

1.00 

I will vary the original design by creating a mechanism to change the 
shutter speed. This will consist of a shutter that can be adjusted and 
therefore, changes the speed. 

No modification of spring 
mechanism 

0.00 

I would use a series of mirrors to prevent any finger shots. Mirrors 
will be used to redirect the line of sight for the user. So instead of 
looking out of a viewer, the user can see exactly what will be showing 
up in the lens. 

Mechanism identified to link rays 
entering lens to viewfinder 

1.00 

After the light passes the lens, but before the shutter, there would 
need to be a device to send an image up to the viewfinder. It couldn’t 
be after the shutter or the only time an image would appear in the 
viewfinder is when a picture was taken. 

No mechanism identified to link 
rays entering lens to viewfinder 

0.67 

I will make the lens that user look through [viewfinder] to be right 
above or even on the camera lens where you take pictures. Then the 
objects that you see through will be the pictures of the objects you 
are taking. 

Suggests lens and viewfinder 
should be superimposed 

0.33 

You need to make the viewfinder bigger. 

No indication of the 
interconnectedness of the lens and 
viewfinder 

0.00 


Table 5. Examples of scored responses to the variant design questions. 


Posttest 2 

Students were presented with images of 18 Fujifilm QuickSnap Outdoor 1000 camera components 
and asked to select from this group all the components that function to prevent the superimpos¬ 
ing of images. This task was scored in the same way as the system decomposition questions. The 
average score of students in the DAA First and Lecture First situations were compared to test for 
condition equivalency at the end of the lab. 

Post Lab Survey 

To establish the goodness of the motivation measures used (students’ perceived sense of learn¬ 
ing, enjoyment derived from engaging in the activity, and helpfulness in preparing them to respond 
to the posttest 1), the interitem consistency reliability was measured by calculating the Cronbach’s 
alpha for motivation elicited by the DAA activity and lecture. In addition, convergent validity was 
tested by correlating the ratings obtained for the DAA activity using the motivation measures with 
the subscale scores from the IMI. 
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It was expected that students would find the DAA activity more motivating than the lecture, and 
to test this hypothesis the mean rating for each element of the motivation measure was calculated 
for each instructional task and statistically compared. Using the appropriate statistical tests, students’ 
responses to posttest 1 were further analyzed to determine if gender, prior experience disassembling 
objects, and motivation had any effect on the outcome. 


RESULTS 

A statistical significance level of a = .05 was set for all statistical tests, and results of tests de¬ 
scribed as significant have a p value that meets this criterion. Statistical tests were conducted using 
the SPSS 16.0 for Windows (November 15, 2007) computer package. 

Motivation 

The three measures of motivation, i.e., perceived sense of learning, enjoyment derived from en¬ 
gaging in the activity, and helpfulness in preparing students to respond to posttest 1, were found to 
have an acceptable interitem consistency. The Cronbach’s alpha reliability coefficient for the measure 
of motivation elicited from the lecture and DAA activity are .775 and .780, respectively. 

To test for convergent validity, the previously identified measures for motivation elicited by the 
DAA activity were correlated with the subscale scores of the Intrinsic Motivation Inventory (IMI), which 
was also used to assess participants’ subjective experience related to the DAA activity. The enjoyment 
element from the motivation measure (M ± SD = 5.87 ± 1.18) and interest from the IMI (5.20 ± 1.23) 
were significantly correlated, r (325) = 0.676, p < .001. There was also a strong significant correlation 
of value from the IMI (4.86 ± 1.23) as was the case for both the learning element from the motivation 
measure (5.30 ± 1.24), r (325) = 0.602, p < 0.001 and the helpfulness element from the motivation 
measure (5.15 ± 1.22), r (325) = 0.545, p < 0.001. 

Using the paired samples t-test on each element of the motivation measure, the mean ratings 
for the DAA task were found to be significantly higher than the mean ratings for the lecture, as 
illustrated in Figure 3. On students’ perceived sense of learning which ranged from nothing (1) 
to a lot (7), the mean rating for the DAA task was 5.30 ± 1.24 (M ± SD), and for the lecture task, 
it was 4.84 ± 1.26, t (324) = 7.580, p < .001. On enjoyment derived from engaging in the activity 
which ranged from strongly disliked (1) to strongly liked (7), the mean rating for the DAA task 
was 5.87 ± 1.18, and for the lecture, it was 5.23 ± 1.31, t (324) = 10.641, p < .001. On helpfulness in 
preparing students to respond to the variant design question in task 5 which ranged from not 
helpful (1) to very helpful (7), the mean rating for the DAA task was 5.15 ± 1.22 and for the lecture 
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Motivation Measures 



Error Bars: 95% Cl 


Figure 3. Motivation measures. 


it was 4.81 ± 1.25, t (324) = 5.263, p < .001. The motivation measures were further supported by 
the comments students provided on the post lab survey. Of the 57 students that provided com¬ 
ments related to the lab, 86% referred to satisfaction or enjoyment. Table 6 shows a sample of 
these responses. 

Learning 
Task 1 (Pretest) 

Out of a possible eight points, the mean pretest score for students in the DAA First sequence 
was 1.01 3 .952 (M 3 5D), and, for students in the Lecture First sequence, it was 1.28 3 1.14. The results 
of the independent t-test, indicate a significant difference in the mean pretest scores of the two 
sequences with students in the Lecture First sequence having an advantage, with respect to prior 
knowledge of the content, to students in the DAA First sequence, t (314) = -2.26 (equal variances 
not assumed), p = 0.024. The results of the pretest, however, indicate an overall low prior knowledge 
of the design of the Fujifilm single-use camera, and if either condition started with an advantage in 
prior knowledge, it was the control condition (Lecture First). 
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Comments 

• This activity was great and I didn’t fall asleep for the first time in a 126 class 

• If this is engineering, I love engineering!!! 

• I liked this lab a lot; it was nice to be exposed to another aspect of engineering. Not just staring at a computer and 
plugging in functions and numbers. 

• I think this was a good activity to learn about the importance of reverse engineering. 

• This class teaches me important engineering skills. 

• I found this activity intriguing. I would like to do more activities like this. 

• This presentation was different and had a new learning experience. 

• Good Program. Would like to see programs like these more often in lab. 

• I liked this lab more than the other labs. 

• Great activity, I am now again considering ME. 

• I had a great deal of fun. I felt that the presenter did a very good job and these presentations have impacted me in a 
very positive way. 

• This was a good activity to do because many people have no experience in this sort of experiment 

• I liked taking the camera apart cause I felt that I had a better understanding of what everything did 

• Reverse Engineering is a good key to learning... 

• I love reverse engineering, it makes learning a lot easier. 

• It’s the most amazing class where we can get a feel of what actually engineering is. 

• This was an amazing lab. I would hope that all labs could be similar to this, instead of doing activities we don’t really 
need in real life. 

• This was a fun lab and I learned a lot about the functions of cameras. 

• This is the most interesting lab yet! It was hands-on, and therefore much more interactive and interesting. 

• I liked this activity because we, as freshman, are searching for hands-on classes, but what we find are classes that 
require reading and tests, not so much tangible stuff. This was a great relief from that monotony 


Table 6. Additional comments from students. 


For task 2, students in the DAA First sequence systematically disassembled the Fujifilm QuickSnap 
Outdoor 1000 cameras while students in the Lecture First sequence viewed a presentation on the 
design of the same camera. These tasks were expected to produce different types of learning which 
students were expected to demonstrate in their responses to the different posttest 1 questions, i.e., 
system decomposition, variant design and camera doctor. 

Task 3 (Posttest 1 - System Decomposition) 

The Cronbach’s alpha reliability coefficient of the 4-question measure of system decomposition 
ability was .544. We suspect that this measure showed low internal consistency at least in part 
because students appeared to interpret the notion of part-function relationships differently. Some 
students appeared to use a narrower definition (only the most immediately critical parts matter 
for the function) while others appeared to use a broader definition (parts that support critical 
parts should be counted). Each of these questions asked students to identify all the components, 
from a selection of 18, that function to allow the described action to occur. The maximum pos¬ 
sible score was 4. An independent t-test revealed that students who did the DAA activity scored 
significantly higher on these questions than the students who had the lecture, t (315) = 3.09 (equal 
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variances not assumed), p = 0.002 (M ± SD = 3.19 ± 0.31 and 3.09 ± 0.27 for DAA First and Lecture 
First respectively). 

Task 3 (Posttest 1 - Camera Doctor) 

The Cronbach’s alpha reliability coefficient of the 4-question measure of camera doctor (defect 
diagnosis) is .551. Again, this low internal consistency on these items suggests that these results 
should be interpreted with caution. For each question students were required to do three things: 
pick one of four user or camera actions that would cause each flawed photograph, select the compo¬ 
nents of the camera associated with the user or camera action that may be defective and provide a 
written description of how the main defective mechanism is most likely responsible for the camera’s 
poor functionality. Using the aggregate of the scores from all the components of each of these 
questions, the independent t-test revealed that there was no difference in performance between 
the students in the DAA First sequence and the students in the Lecture First sequence, t (323) = 
.538, p = .591 (M ± SD = 6.07 ± 1.78 and 5.97 ± 1.73 for DAA First and Lecture First, respectively). 

Each of the three camera questions increased in degree of specificity from first picking the 
camera function, to then selecting relevant parts, to finally identifying and explaining how the criti¬ 
cal mechanism malfunctioned and resulted in the poor quality of the photo. This third part of the 
camera doctor question, was expected to be most sensitive to students’ understanding of specific 
part-function relationships. Thus, a separate post-hoc analysis was conducted on just these scores. 
The independent t-test revealed that students in the DAA First sequence performed significantly 
better on these questions than students in the Lecture First sequence, t (323) = 2.026, p = .044 (M± 
SD = 1.26 ± 0.90 and 1.06 ± 0.87 for DAA First and Lecture First, respectively). The highest possible 
aggregate score is 3. Figure 4 shows a comparison by condition on this measure. 

Task 3 (Posttest 1 - Variant Design) 

The two variant design questions asked students to describe how the original design of the 
camera can be modified to achieve a new functionality. For the first question the new functional¬ 
ity was three different shutter speeds. Possible scores for this question were either 1 for responses 
that described a modification to the spring system, or 0 for responses that did not. Among the 
162 students that did the DAA activity first, 110 received a score of 1. Among the 163 students that 
had the lecture first, 49 received a score of 1. Based on the results of the Pearson’s chi-square test, 
students in the DAA First sequence performed significantly better on this question than students 
in the Lecture First sequence, x 2 0 , N = 325) = 46.557, p < .001. 

Given that the system decomposition questions preceded the variant design questions, further 
analysis attempting to control for prior knowledge was possible. Students who noticed that the 
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Figure 4. Camera doctor: comparison on the description of the primary defective 
component. 


Camera Doctor - Malfunction Description 



DAAFirst LECFirst 


spring was involved in taking a picture (i.e., got credit for that component in the system decom¬ 
position question), had relevant prior knowledge for adapting the spring mechanism in the variant 
design question about adding variable shutter speeds. More students who did the DAA activity first 
(DAA students) noticed the spring feature in the system decomposition question than students 
who had the lecture first (Lecture students), % 2 (1, N = 325) = 17.319, p < .001. Examining only the 101 
DAA students and the 64 Lecture students who showed the relevant prior knowledge of the spring 
mechanism, DAA students still showed significantly more transfer to the variant design task that 
required adapting the spring mechanism, % 2 0, N = 165) = 25.224, p < .001. This suggests that DAA 
activity was significantly better at helping students adapt their prior knowledge to solve a new 
design problem, even after attempting to control for relevant prior knowledge. 

In the second variant design question, the new functionality was a viewfinder that can be used to 
see any object blocking the lens. Possible scores for this question were either 1 for responses that 
identified an appropriate mechanism for directing the rays of light entering the lens to the viewfinder, 
2 / 3 for responses that indicated that a mechanism is required to direct the rays of light entering the 
lens to the viewfinder, without identifying the mechanism, V z for responses that suggest the lens 
and viewfinder should be superimposed, and 0 for responses that do not mention or suggest the 
interconnectedness of the lens and viewfinder. Among the 162 students that did the DAA activity 
first, the average score (± 5D) is .504 ± .480. Among the 163 students that had the lecture first, the 
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Variant Design - Viewfinder / Lens Question 

100- 
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■DAA First 
□ LEC First 


Figure 5. Distribution of scores for the variant design questions. 


average score (± SD) is .391 ± .460. Based on the results of the Mann-Whitney U test, students in the 
DAA First sequence performed significantly better on this question than students in the Lecture First 
sequence, (U = 11596.000, /7 1 = 162, n 2 = 163, p = .036 two-tailed). Figure 5 shows the distribution of 
scores for both variant design questions, and Figure 6 shows the relationship between prior relevant 
part-function knowledge and the adaptation of that knowledge in the variant design question. 



DAA First Lecture First 


■ #of students that 
demonstrated prior 
knowledge of the 
involvement of the spring 
in the shoot action 
(based on system 
decomposition question) 

■ # of students with prior 
part-function knowledge 
on the spring-shoot action 
that adapt the spring in 
the variant design 
question on variable 
shutter speeds 


Figure 6. Relationship between prior relevant part-function knowledge and the 
adaptation of that knowledge in the variant design question. 
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Task 5 (Posttest 2) 

The posttest 2 question asked students to choose from a selection of 18, all the components that 
function to prevent the superimposing of images. Scores for this question could range between 
0 and 1. Based on the results of the independent t-test, there was no significant difference in the 
performance of students in the DAA First and Lecture First sequence, t (323) = .341, p = 0.733 (M ± 
SD = 0.81 ± 0.13 and 0.80 ± 0.11 for DAA First and Lecture First respectively). These results establish 
condition comparability at the end of the lab. 

Effect of Gender, Prior Disassembly Experience, and Motivation 

Secondary analyses focused on testing for potential relationships among gender, prior disas¬ 
sembling experience and motivation, as well as potential relationships between students’ ability to 
transfer knowledge from the DAA task, which was evident in the posttest 1 responses of students 
in the DAA First sequence and any of the demographic, background, and motivation measures col¬ 
lected in the post lab survey. A relationship was found between gender and prior disassembling 
experience and gender and perceived competence on the DAA activity. On a seven-point scale that 
ranged from 1 (no experience) to 7 (extensive experience), the average prior experience reported 
by men was 4.49 ± 1.68 (M ± SD), and for women it was 2.78 ± 1.63. The results of the t-test indicate 
that this difference is significant, t (323) = 7.46, p < .001. On a similar scale the average competence 
reported by men (± SD) was 5.21 ± .923, and for women (± SD) it was 4.78 ± 1.09. The results of the 
t-test indicate that this difference is also significant, t (92) = 2.93 (equal variances not assumed), 
p = 0.004. Prior disassembling experience (M 3 SD = 4.14 ± 1.81) and perceived competence on the 
DAA activity (5.12 ± .974) are significantly correlated, r (325) = 0.362, r 2 = .131, p < .001. Beyond the 
aforementioned, no other relationships were found among the variables (gender, prior disassembling 
experience and motivation) examined, and none of them accounted for any significant variability in 
posttest 1 responses for students in the DAA First sequence. 


DISCUSSION AND CONCLUSION 

This study, like the previous experiment conducted by the authors (i.e., Dalrymple et al., 2011), 
compared a DAA activity to a control in terms of motivation and transfer. In this study the control, a 
lecture, provided better comparability in terms of learning objectives and transferable content. Both 
the lecture and the DAA activity were structured to help students learn about the DfE design prin¬ 
ciples and the part-function relationships embodied in the Fujifilm single-use camera. With the DAA 
activity, students disassembled the camera and analyzed its components to discover their function 
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and interconnectedness, while the lecture presented similar content with the use of a multimedia 
PowerPoint presentation (removing the manipulation and discovery aspects of the DAA process). 
The hypotheses remained somewhat consistent, particularly given the experimental findings on the 
deficiencies of lecture with respect to promotion of thought and inspiring interest in a subject (Bligh, 
2000). It was expected that on measures of motivation, the DAA activity would be rated higher than 
lecture; on measures of learning where the task required students to recall the part-function relation¬ 
ships explored in both instructional approaches, the DAA activity and lecture would look equivalent; 
and on measures of deeper understanding, where the tasks moved away from recalling part-function 
relationships and towards diagnosis and redesign, the DAA activity would outperform lecture. 

With respect to motivation, the results were consistent with previous findings. Notwithstanding 
students’ high ratings for lecture (5.23, 4.84, and 5.15 on a seven-point scale for enjoyment, learn¬ 
ing, and helpfulness, respectively), which were greater than those for both comparative tasks in 
the previous experiment, the DAA activity proved to be significantly more motivating on all three 
measures, regardless of the order in which students experienced the activities. The motivation mea¬ 
sures continued to produce alpha coefficients greater than 0.75 and showed significant correlations 
to the peer reviewed Intrinsic Motivation Inventory (IMI); further validating these findings. 

Regarding learning, three different posttest 1 questions were used to characterize the potential 
differences between lecture and the DAA activity. The three question types varied in context, rela¬ 
tive to the instructional/initial learning tasks. The system decomposition questions, most closely 
related to the context of initial learning, tested for students’ ability to recall the associations between 
camera parts and their function. The camera doctor questions asked students to diagnose defects 
in a camera, based on the analysis of photographs. The question retained some aspects of the part- 
function association and advertently tested for recall; however it deviated into the realm of novel 
problem solving, asking students to diagnose the presented symptoms and generate hypotheses 
about the source of the malfunction (Jonassen, 2000). The variant design question deviated most 
from the context of initial learning, testing for the greatest distance of transfer. It required students 
to modify the current camera design to achieve new functionality. Design problems are the most 
complex and ill-structured kinds of problems (Jonassen, 2000). 

The results showed that the DAA activity had advantages over lecture in terms of recall and 
transfer. Students who did the DAA activity scored higher than those who had the lecture on the 
system decomposition and variant design questions. On the camera doctor questions there initially 
appeared to be no difference; however, once the component-function part of the question was 
removed from the analysis, the DAA activity showed advantages over the lecture. Students that 
did the DAA activity were better able to generate plausible hypotheses about the reason for the 
malfunction. It is important to note that the DAA activity lead to greater transfer as observed in 
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the initial study, without the added cost of additional instruction or worse basic understanding. It 
is also impressive that students with one exposure to one camera were able to notice and adapt 
its features to develop a new design and diagnose defects in a camera after viewing flawed pho¬ 
tographs. In other literature on transfer, it often takes multiple examples or cases for students to 
develop a working schema (Gick & Holyoak, 1983) or show adaptive expertise (Sears, 2006). In this 
sense, being able to adapt knowledge after one exposure is impressive and reveals a potential key 
advantage of the DAA process (iteration of observation and follow-up probing). 

The learning and transfer observed from the DAA activity was not impacted by any of the vari¬ 
ables measured in the post lab survey. None of the demographic, background, and motivation data 
accounted for any significant variability in students’ ability to learn or transfer knowledge from the 
DAA activity. Although women reported having significantly less prior disassembling experience 
than men, similar to the findings of study 1, and a lower perception of their competence on the DAA 
activity than men, they performed equally to men on measures of learning and transfer. 

Both studies conducted by the authors involved a camera, a tangible and predominantly me¬ 
chanical device, as the artefact under investigation. Would the observed benefits remain true for 
other types of engineering artefacts? The DAA framework has already begun to show promise at 
cultivating student’s adaptive expertise in engineering, i.e., ability to apply knowledge effectively 
to novel problems. These early findings push for further examination to test the generalizability of 
the claims. New types of engineering artefacts, beyond those that are predominantly mechanical 
and/or, tangible, need to be assessed. 
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